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Pathways to the Future: Nga Huarahi Arataki, the ten year strategic plan for early childhood 
education (ECE) in New Zealand, identified reviewing the delivery of professional 
development as one strategy for ‘promoting the effective delivery of Te Whariki 
(Ministry of Education, 2002, p. 15) within the overall goal of improving quality within 
ECE services. This paper reports on aspects of a national evaluation of ECE 
professional development undertaken for the Ministry of Education in 2005—06. The 
paper describes the multi-method approach taken, and highlights findings in two key 
areas: issues around access to professional development and barriers to practitioners 
engaging in PD, and the effectiveness of professional development programmes in 
supporting and sustaining shifts in pedagogical practices underpinned by Te Whariki. 
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INTRODUCTION 

Improving the quality of early childhood education (ECE) services is a key goal of New Zealand’s 
strategic plan for ECE, Pathivays to the Future: Nga Huarahi Arataki (Ministry of Education, 2002). 
One strategy identified to achieve this goal is to ‘promote the effective delivery of Te Whariki (p. 
15), the national curriculum for early childhood education, covering all licensed services for 
children 0-6 years (Ministry of Education, 1996). Since Te Whariki appeared in draft form in 1993 
the Mnistry of Education (MOE) has funded professional development (PD) programmes 
through independent contractors to support its implementation. Specific contracts were let for this 
purpose alongside general professional development programmes from 1995-1998. Since 1999 all 
ECE PD contracts have been required to support the teachers’ use of Te Whariki. Funding for 
these programmes has been based on services being able to access programmes at least once 
within a three-year cycle. 

A review of the effectiveness of ECE PD programmes in supporting the delivery of Te Whariki 
was identified as an action within the Strategic Plan (Ministry of Education, 2002), and in 2005 
Victoria University was contracted to undertake this evaluation. Earlier reviews of ECE 
professional development contracts (Duthie Educational Consultancy, 1996; Gaffney, 2003) 
addressed MOE-funded professional development programmes only, seeking the perspectives of 
services and teachers engaged in these together with the views of providers delivering such 
programmes. In contrast, this evaluation took a broader stance by evaluating the delivery of MOE- 
funded PD contracts within the wider context of ECE professional development generally, and 
employed a more extensive evaluation methodology. Thus, the size and scope of this evaluation 
went beyond previous evaluations of ECE professional development in New Zealand. 
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In the context of this evaluation, professional development was conceptualised as formal 
teacher learning activities that involve external facilitation (Buysse, Winton, and Raus, 2009), rather 
than the informal learning that may occur through collegial discussions and in self-study activities. 
The Ministry of Education identified three key areas of interest that framed the evaluation: an 
analysis of current PD provision, delivered both through MOE-funded programmes and through 
other providers; an evaluation of the effectiveness of PD in supporting and sustaining changes in 
practice that positively impact on children’s learning outcomes; and the identification of strategies 
for improving MOE-funded PD. The findings reported here draw on data from the first focus area 
to discuss access to professional development and barriers to practitioners engaging in PD, and 
from the second focus area to examine the effectiveness of PD programmes in supporting and 
sustaining shifts in pedagogical practices underpinned by Te Whariki. 

Effective professional development 

Despite significant research attention towards teachers’ professional development and learning, 
debate continues as to what constitutes effective professional development (Borko, Jacobs, 
Eiteljorg and Pittman, 2008; Buysse et al., 2009). Several factors have been identified as 
contributing to effective PD, including on-going engagement (Guskey, 2000) within a community 
of practice (Bayes, 2005; Jordan, 2003; Webster-Wright, 2009) which has access to ongoing 
external support (Depree and Hayward, 2000; Lidington, 2000). 

Efforts to understand the empirical research on teacher learning have resulted in two best 
evidence syntheses in New Zealand focused on professional development and learning in early 
childhood settings (Mitchell and Cubey, 2003) and in the school sector (Timperley, Wilson, Barrae 
and Fung, 2007). Mitchell and Cubey (2003) identify eight characteristics of effective PD linked to 
enhanced pedagogy and children’s learning in early childhood settings. Such PD draws on 
practitioners’ existing knowledge and understanding and assists awareness of their own thinking 
and actions, as well as positioning them as agents investigating their own pedagogy, analysing data 
from their own settings, and engaging in critical reflection. In addition effective PD provides 
theoretical and content knowledge, supports inclusive practices and changes practitioners’ beliefs, 
practices and attitudes. 

Timperley et al.’ s (2007) synthesis of empirical literature identified twenty-two factors located 
within four core aspects—context, content, activities and learning processes—of the professional 
development process that have a positive effect on students learning. The existence of many of 
these factors (for example, extended time for opportunities to learn or participation in a 
community of practice) was necessary but of themselves did not automatically lead to effective PD. 

Professional development is more likely to be effective when participants are active and 
purposeful (Hampton, 2000), when teachers have control over the PD programme (Blenkin and 
Kelly, 1997) and see it as part of their ongoing, lifelong learning (Gaffney, 2003). Management 
support for PD is an important influence on the success of programmes (Gaffney, 2003), 
particularly where this enables teachers to implement change (Lidington, 2000). 

The quality of facilitation within PD programmes is also important. Mitchell and Cubey 
identified several attributes of effective facilitators including having strong theoretical, content, and 
pedagogical knowledge, and being able to understand and to challenge practices and ideology that 
disempower the interests of children and families (Mitchell and Cubey, 2003, p.xiv). The 
importance of an outside facilitator has been noted in several New Zealand studies focused on the 
implementation of The Quality Journey (Ministry of Education, 1999) where participants reported 
that they would not have engaged with the document without such support (Depree and Hayward, 
2000; McLauchlan-Smith et al., 2001; Wansbrough, 2003; White; 2003). Similarly, Lidington (2000) 
found external support was important for participants’ trialling implementation activities. 
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Teacher attitudes influence the learning of children from diverse families (Mitchell and Cubey, 
2003), especially when these attitudes result in lowered expectations of their learning. Professional 
development that challenges such assumptions (Alton-Lee, 2005; Bishop et al; 2003; Timperley, 
Phillips, Wiseman and Fung, 2003) helps teachers move to a credit view of children and their 
families. The collection and analysis of data, often supported by an ‘outsider’ (the facilitator) 
(Timperley et al., 2003) able to confront teachers with data from video and assessment evidence, 
enables teachers to critique their practice and encourages pedagogy that better builds upon 
children’s strengths and home experiences. 

Background to the New Zealand context 

In the evaluation reported here, professional development refers to programmes offered to 
teachers and educators in licensed, chartered ECE services 1 in New Zealand, often referred to as 
inservice professional learning, and does not include teacher education programmes (preservice). 
Most programmes are delivered by MOE-funded contractors, selected through a contestable 
process. Participation in the programme is generally at no cost to the participants. Fourteen 
providers of ECE PD programmes were contracted to the MOE, including Universities, Colleges 
of Education, private organisations, and independent ECE organisations, at the time of this 
evaluation. 

A variety of delivery approaches were used by the MOE-contracted providers in their 
programmes. Almost all providers offered whole-centre programmes where all teaching staff from a 
service participated in a common PD focus, and most used cluster-group programmes where 
practitioners from several services met together for workshops followed by individual sessions in 
each service setting. Opportunities for individuals to attend seminars, short courses or networks 
usually constituted a minor component of the programmes offered by the MOE-funded providers. 
Professional development sessions were led by facilitators who used a range of approaches, 
including meeting with staff at their workplace, observing teacher practices during the programme, 
and facilitating workshops, seminars and networks. 

Beyond MOE-funded PD programmes, an array of other professional development 
opportunities is available. Independent (including private and not-for-profit) organisations and 
government agencies offer PD opportunities to the wider ECE community, often at a cost to 
participants, and many umbrella organisations organise and provide PD for staff employed in their 
services. Attendance at annual conferences offered by early childhood organisations as well as New 
Zealand’s 4-yearly Early Childhood Convention is increasingly popular with teachers, particularly 
where these address specialist or philosophical PD needs (e.g., Montessori or Steiner teacher 
conferences). More broadly based research conferences are also attracting some teachers. The 
Teachers Refresher Course Committee (TRCC), a MOE-funded group of teacher representatives 
across the education sector, organises courses that ECE teachers are able to access. These 
residential courses are usually 3-5 days in length and allow for in-depth engagement with a topic 
and networking with teachers who have similar interests (Thornton, 2002). 

Currently, not all teachers in New Zealand ECE services are qualified and registered. Thus, 
professional development is not solely a post-qualification activity as it is in the school sector and 
participants in early childhood PD programmes often include a mix of trained and untrained 
personnel from services. At the time of the evaluation there was a requirement for a person with 
responsibility for the teaching and learning in each ECE service to hold an ECE teaching 
qualification and be a registered teacher. For kindergarten services, all teachers were required to be 
qualified and registered. 
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Focus of New Zealand ECE PD programmes 

MOE-funded professional development programmes have explicitly encouraged teachers to 
engage with Te Whariki and to explore the complexities of pedagogical practices demanded by the 
curriculum’s theoretical underpinnings. Early PD programmes, established to support the 
introduction of Te Whariki, focused on exploring the curriculum document and understanding its 
principles, strands and goals. Initial concerns (e.g., Cullen, 1996) that teachers did not recognise the 
shift in pedagogical thinking inherent in the curriculum document, interpreting it instead as 
confirming existing developmentally-based practices, have persisted. Meade (2000) has noted the 
continuing dominance of Piagetian theory whilst Anning, Cullen and Fleer (2009) argue that ‘it has 
not been easy for educators to appreciate the complexity of the Te Whariki curriculum, or to take 
the more proactive teaching role envisaged by a socio-cultural philosophy’ (p. 20). More recent 
programmes have shifted from the earlier focus on the structure of the curriculum document with 
teachers now scrutinising their own practices in order to improve the quality of interactions with 
children and their families. 

More recently, programmes have involved teachers in scrutinising their own practices using a 
sociocultural framework, within the context of assessment for learning, planning and evaluation. 
The early childhood assessment exemplars, Kei Tua o te Pae (Ministry of Education, 2005 were 
newly introduced to the sector around the time of the evaluation (2005-06) and have significantly 
influenced both PD programmes and teacher practices. 


EVALUATION METHODOLOGY 

Evaluating the effectiveness of professional development programmes is challenging. Evaluations 
undertaken at the end of a programme, regardless of design, have a limited ability to measure 
changes in practice as consolidating and assimilating new practices into a service’s culture may take 
a year or two to show effect (Gould, 1999; Guskey, 2000). In this external evaluation, two factors 
influenced the research design: firstly, the evaluation was focused on recently completed 
professional development programmes where long-term evidence of change was not available, and 
secondly, the project parameters did not allow for direct observation by the evaluators of 
participants’ practices pre- and post-involvement in professional development activities. Within 
these constraints a multi-method evaluation was designed, using a wide range of sources to gather 
data, as both a process evaluation (focused on the PD delivery processes) and a formative 
evaluation (focused on how the delivery of PD could be improved and enhanced to meet MOE 
strategic goals). An in-depth literature review informed the evaluation design and development of 
research instruments. 

A national survey, comprising 47 quantitative and two qualitative questions, was sent to a 
randomly selected sample of 1439 (46%) licensed and chartered early childhood services across 
New Zealand, excluding kahanga reo which were not included in the evaluation. Over-sampling was 
used to ensure statistically valid numbers of specific services in each geographical region of the 
country. A 52% (N=743) return rate of the questionnaires was achieved. Further in-depth 
qualitative data was gathered through six group interviews with teachers held across New Zealand, 
designed to ensure that perspectives from diverse services and particular regional groups were 
heard. Telephone interviews were conducted with all fourteen MOE-funded PD contract directors 
for the 2004/05 contract years. Requests for a telephone interview were made to the major 
national ECE umbrella organisations with key personnel in six organisations agreeing to be 
interviewed. Six group interviews were conducted with personnel from the MOE Plead Office and 
regional offices. Final milestone reports from each of the 2004 contractors were also analysed. 
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The survey of early childhood services and teacher group interviews required teacher self- 
reports of their PD experiences and of the changes that had resulted from their engagement in PD. 
The limitations of self-report data were offset somewhat by the document analysis which enabled 
providers’ observations of changes in practices to be considered, and by the interviews with 
contract directors, MOE officials, and umbrella organisations. The latter interviews, in particular, 
offered insights into the changes that organisational leaders had observed in teacher practices 
within their organisations and which some noted were reflected in external evaluation 11 reports on 
the services. 

The use of a mixed-method approach (Green, 1998) enabled the inclusion of both quantitative 
and qualitative data, and allowed for the triangulation of data from different methodologies in 
order to increase the validity of interpretations. Wherever possible, results from quantitative and 
qualitative sources were analysed for triangulation. Data from the quantitative source 
(questionnaire) were used as triggers for exploration and explanation in the interviews. In this way 
the quantitative and qualitative methods were interactive and supportive in uncovering 
perspectives on ECE PD experiences and provision in New Zealand. A fuller description of the 
methodology and the evaluation tools used can be found in the full report on the website 
www.educationcounts.govt.n 2 /publications/ece (Cherrington and Wansbrough, 2007). 

As Victoria University was a contractor delivering one of the MOE-funded PD contracts, 
significant efforts were made to avoid perceptions or actual conflicts of interest, including using 
members of the evaluation team without ECE backgrounds to conduct those interviews where a 
perceived conflict might have arisen. Statistical analysis of the quantitative survey data was 
undertaken using SPSS whilst the qualitative data from the interviews and survey were coded and 
analysed using QSR N6. 

This article draws on quantitative data from the survey, together with qualitative data from the 
interviews with teachers, contract directors, MOE officials and umbrella organisations and from 
the document analysis to discuss access to professional development and barriers to practitioners 
engaging in PD, and to examine the effectiveness of PD programmes in supporting and sustaining 
shifts in pedagogical practices underpinned by Te Whariki. 


ACCESS TO ECE PROFESSIONAL DEVELOPMENT 

Data concerning which services were accessing PD nationally, by region and by service type were 
gathered from the survey and from provider milestone reports. Inconsistent reporting styles in the 
milestone reports meant that this information was not uniformly provided, and thus the survey of 
services provided the most accurate indication of services’ access to PD. Survey data indicate that 
the proportion of PD accessed through MOE-funded contracts varied considerably at the national 
level. Whilst 26% of respondents received more than three-quarters or all of their PD through 
these contracts, 38% received less than one-quarter or none at all. When these data were cross 
tabulated by service type they revealed that only 12% of kindergartens accessed three-quarters of 
their PD from MOE-funded programmes compared to 25% of home-based services, 36% of 
education and care services and 37% of playcentres. 

When asked how difficult it was to access PD, nationally just over one-quarter (28%) of 
services found it very difficult or difficult to access any professional development through MOE- 
funded contracts; similarly 29% of respondents also found it difficult to access PD through non 
MOE-funded providers. In contrast to these national results, the data by region revealed wide¬ 
spread variations with between 21-75% of services reporting that they found it very or quite 
difficult to access MOE-funded PD. These results did not show clear patterns of difficulty in 
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access by type of region: whilst respondents in five regions without large cities (West Coast, 
Northland, Tasman/Nelson/Marlborough, Gisborne and Bay of Plenty) had more difficulty so 
too did two districts with major cities (Waikato and Canterbury). Those with low rates of difficulty 
also varied in terms of population density: Auckland and Manawatu/Wanganui have large 
population centres whilst Taranaki, Hawke’s Bay and Southland have smaller cities. However, 
across the country those services who identified themselves as rural in the survey reported greater 
difficulty in accessing PD. Data from the national survey, together with data from the interviews 
with providers, umbrella organisations and teachers suggests that rural and isolated services find it 
more difficult to access PD and face greater costs in terms of travel and time than do other 
services. In addition, some services experienced isolation due to their special character (e.g., 
Montessori services) which created difficulties in accessing appropriate PD. 

Due to the inconsistent reporting styles noted above, analysing service participation in PD 
programmes through data presented in provider milestone reports relied at times on assumed 
groupings. For example, where a provider offered programmes using a cluster-group model and 
did not include the actual number of services enrolled, a notional figure of four services per cluster 
was adopted. Cluster group enrolment figures (whether actual or notional) were then added to the 
numbers of services enrolled in individual whole-centre PD programmes in order to identify the 
overall number of services accessing PD within specific regions. Data from the milestone reports 
suggest that major metropolitan areas fared worst in terms of accessibility with 17% of services in 
the Auckland and Wellington regions accessing PD programmes for whole-centre or cluster-centre 
programmes followed by the Canterbury (20%) and Waikato (23%) regions. Some areas classed as 
more geographically isolated (e.g., Southland, Otago, Northland) had higher rates of involvement 
in whole-centre or cluster-centre PD programmes, with between 32% and 37% of services in these 
regions accessing this MOE-funded PD. Inconsistencies between the data from these milestone 
reports and the survey data, are due, in part, to the milestone reporting approaches. In addition, 
services’ perceptions may not completely match actual difficulties in accessing whole-centre or 
cluster-centre PD. 

The survey data also revealed considerable variation in access to PD by service type. 
Kindergartens (37%) and home-based services (36%) reported having the greatest difficulty in 
accessing MOE-funded PD in comparison with education and care services (23%) and playcentres 
(30%). Data from the provider milestone reports reflect these results: when analysed by service 
type, the milestone reports revealed that the number of kindergartens (9%) and home-based care 
services (11%) accessing this form of PD were very low compared to education and care, 
playcentre, and Maori immersion services (26%). The very high rates of involvement of Pasifika 
services (70%) in whole-centre PD are skewed by the limited numbers of Pasifika services overall 
and the delivery of a specific contract focused on these services in 2004. 

Participation in non MOE-funded PD 

The provision of professional development opportunities by diverse organisations outside of the 
MOE-funded PD contracts has grown steadily in recent years. Engagement in such PD is most 
likely to be through participation in short courses (74%) and conferences/symposia (62%). The 
lower numbers of survey respondents participating in TRCC (28%) and Rural Education Action 
Programme (REAP) courses (19%) indicates their more limited availability nationally and the more 
specific nature of these programmes. Across these four types of non-MOE funding PD, 
respondents’ participation was influenced by their interest in the topic or focus, the quality of 
speakers, and the opportunities for networking. 
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Determining engagement in PD 

Both the qualitative survey questions and the interviews revealed that access to MOE-funded PD 
is influenced by the degree to which the service (or its umbrella management organisation) actively 
seeks PD that is tailored to its needs or, alternatively, waits for PD information to come from the 
contracted provider(s) in their area. With funding designed to enable one-third of services to 
access PD in a calendar year, reticent services are likely to have reduced, or no, access to MOE- 
funded PD. 

Teachers responding to the survey were asked to identify from a list of six possible choices 
how decisions about attendance in PD were generally made. Overwhelmingly, decisions are made 
by teachers rather than imposed by management or external agencies such as the Education 
Review Office. Elowever, significant variations exist by service type as to how decisions are made 
even when these are made by teachers. Kindergarten teachers were much more likely than 
respondents in other services to make decisions about their PD as a result of their appraisal plans 
whilst home-based and playcentre respondents were most likely to make decisions based on an 
individual’s spontaneous request. Education and care services were equally likely to make decisions 
based on the spontaneous requests of individual teachers and in accordance with the service’s 
vision and goals. The timing of the information about available MOE-funded PD was identified in 
the teacher group interviews as an aspect that could be improved to facilitate more deliberate 
connections between appraisal processes and planning PD requirements. 

Interview data revealed the importance of a continuum of engagement in PD that had at one 
end individuals participating in one-off events such as short courses, through to several staff 
attending such events, through to whole-centre PD and on to cluster whole-centre PD where 
practices are examined with colleagues from other services. Ensuring PD provision along such a 
continuum is supported by Guskey’s (2000) recommendation that PD plans are based on a 
combination of models. Sands (2005) illustrated this shift in engagement when highlighting how 
her centre had moved from perceiving PD as something to be passively involved in to feeling 
empowered to determine the PD process. Providing a variety of PD models enables teachers to 
access PD that fits with their professional and personal lives (Jordan, 2003; Mepham, 2000). 

Barriers to PD participation 

The evaluation sought to ascertain what factors created barriers to teachers’ participation in MOE- 
funded PD. Several questions within the survey teased out respondents’ views on issues previously 
identified in the literature as potential barriers (Baker and Lorrigan, 2000; Gaffney, 2003; Irvine 
and Lovatt, 1996), including the cost of participation. A significant barrier for almost one-third 
(31%) of services was the cost of engaging in PD with a further 45% reporting that costs impacted 
somewhat on their participation. Kindergarten, playcentre and home-based service respondents 
reported being more affected by cost than did education and care services. 

Interview data indicate that the costs of participating in PD included travel, relievers, and 
reimbursement to staff attending sessions after hours, beyond any costs directly associated with 
non-MOE funded PD (such as course fees, conference registration, and accommodation). 
Conferences and other such opportunities were frequently accessed on an individual basis only and 
were often beyond the economic means of services. Teachers generally found it easier to attend 
conferences or other PD opportunities hosted by umbrella organisations as these can be tailored to 
the services’ operating patterns. 

In addition to asking respondents about the impact of cost, the survey also asked respondents 
to rate a number of barriers to engaging in effective PD. The most significant barriers to 
participation were identified as staff workloads, obtaining relief teachers, difficulty in finding time 
to attend, not being able to include the whole team, and unsuitable times for meetings. These data 
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were supported by the qualitative interview data which suggest that the barriers that impact on 
participation in PD are complex and interwoven. Interviewees identified the need for teachers 
within the service to complete their initial teacher education programme in order to meet 
qualification requirements as the biggest impact on the ranking of the three highest ranked 
barriers, as the demands of completing their study mean that many staff were tired and unwilling 
to commit to further PD. 

The negative effect of the shortage of qualified relievers, resulting in qualified teachers being 
unable to take time away from the service to pursue PD, emerged through the interview data 111 . 
Qualified or senior teachers in services reported curtailing their own PD due to a lack of trained 
relievers and the need to maintain a qualified staff count for funding purposes while untrained 
teachers were able to be more easily released. 

Other barriers to participation identified through the interviews include the shortages of 
qualified staff, leading to high rates of staff turnover in some areas; the impact of travel times (for 
urban as well as mral services); dysfunctional teams with weak organisational cultures; and a lack of 
non-contact time for teams to engage in PD opportunities together. 

Effectiveness of ECE Pd programmes in supporting and sustaining shifts in pedagogical 
practices 

Meaningful changes in teacher discourses and practices around curriculum and assessment and 
planning were reported through both the survey and the interviews with practitioners. A very high 
number of survey respondents (85%) believed that their practices had changed a great deal or a fair 
amount as a result of their engagement in PD. When asked to assess the effectiveness of their 
recent PD, similarly high ratings were given for three areas: improving children’s learning outcomes 
(87%), increasing teachers’ content knowledge (85%) and improving their services’ teaching capability (83%). 
Furthermore, a question asking respondents how far their service had progressed in a number of 
areas revealed that the top three rated areas of change concerned pedagogical practices: improved the 
quality of the education practice of service (86%), changed the approach of the service to teaching and learning 
(80%), and increased the pedagogicalj theoretical knowledge of staff in the service (79%). Such findings indicate 
that MOE-funded professional development programmes are making progress in addressing the 
concerns of writers such as Cullen (1996), Meade (1990) and Anning, Cullen and Fleer (2009) 
regarding practitioners’ understandings of the theoretical underpinnings of Te Whariki. 

However, while the above results might be expected given their strong links to PD contract 
priorities, lower ratings regarding the effectiveness of PD programmes in understanding and celebrating 
diversity (68%) and encouraging bicultural understanding (62%) are of concern, given the need for 
services to be effectively working with children and families /whanau from diverse backgrounds 
(Alton-Lee, 2003). Similarly, only 59% of respondents felt that they had become more sensitive to 
bicultural issues as a result of participating in PD and 56% felt that they had become more aware 
of, and able to work with people from diverse backgrounds. Survey data also revealed that services 
do not prioritise PD that focuses on bicultural practices or on diversity. 

Interview respondents were asked about the effectiveness of MOE-funded PD programmes in 
supporting and strengthening bicultural practices. The overarching theme that emerged from these 
data is one of variability in terms of commitment from services to engage in PD on biculturalism; 
provider capability and the quality of programmes offered in this area; and resourcing. The 
interview data also indicates a lack of consensus about what responding to diversity entails. Whilst 
some interviewees interpreted diversity through a cultural lens others saw it as responding to the 
personal circumstances of every family and child. Gifted and talented children were never 
mentioned, and there was a gap in the conversation concerning children with special needs. 

When asked how well their service’s PD had assisted them in obtaining evidence of enhanced 
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learning outcomes for children as related to Te Whariki most respondents reported that their PD had 
assisted them very well (32%) or quite well (56%). When subsequently asked whether they had been 
introduced to useful approaches to assessing children’s learning as part of their PD programme, 
more than half the respondents (56%) indicated that they had been introduced to useful 
approaches, while a further 36% felt they had to some degree. 

From a given list survey respondents were asked to identify how they were able to recognise 
that their PD had been effective. Taking the top two ratings on a four point Likert scale the 
highest rated sources of evidence were observable improvements in children’s learning outcomes (71%), 
observations (67%) and parents/whanau feedback (63%). Respondents rated several indicators poorly as 
sources of evidence of change in their practices: evaluation form at the end of the PD programme (10%), 
feedback from facilitators (16%), achieving previously set indicators (19%) and setting goals to measure against 
(19%). Interestingly, these latter indicators are similar to those identified in milestone reports as 
being used by providers to measure the effectiveness of their programmes: completion of mid- and 
post-programme evaluations against the initial needs analysis; revisiting progress indicators or action plans regularly 
throughout the programme; facilitator observations of service’s!teachers’ practices; reflective discussions between 
facilitators and participants; use of rating scales to measure progress', and satisfaction surveys completed by the 
service! teachers. 

These results indicate a mismatch between sources of evidence valued by respondents and 
those used by providers to report on progress to the Ministry. The latter sources attend to changes 
in teacher practice rather than the impact of PD on children’s learning as well as reflecting the 
reporting demands inherent in contract arrangements. In contrast, those sources of evidence 
valued by respondents may emerge over a time period beyond that of the actual PD programme, 
providing evidence of sustained change in teacher practices in line with Guskey’s (2000) position 
that evaluations should not be limited to immediately following the conclusion of the PD 
programme. 


LIMITATIONS TO THE EVALUATION 

As noted in the discussion of methodology above, there are recognised limitations in relying on 
self-reports of teachers’ professional development experiences, and these are acknowledged here. 
The use of self-report methodology is, however, appropriate when gathering data about people’s 
perceptions and feelings (Howard, 1994; Spector, 1994). Short-comings in its use can be offset to 
some extent by the inclusion of additional data sources to improve validity. The inclusion of 
interviews with providers, umbrella organisations and MOE officials, together with the analysis of 
milestone reports, was intended to enable triangulation with self-report data gathered through the 
survey and group interviews, in order to strengthen validity. The large sample of services surveyed, 
resulting in 23.7% of the country’s ECE services (excluding kahanga red) responding, also 
strengthens the validity of the results. 

The second limitation noted is the timing of the evaluation in relation to the PD programmes. 
Firstly, the focus on PD completed within the 18 month period preceding the evaluation meant 
that there was no opportunity for the evaluators to directly observe programmes or to gather data 
on teachers’ practices prior to the PD intervention. Secondly, whilst the evaluation had the benefit 
of gathering data at a later point than did the provider evaluations, the timing prevented close 
attention to the sustainability of changes in practices over time. 
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CONCLUSION 

The results of this evaluation highlight the complexity and diversity of the ECE sector in New 
Zealand and the challenge of providing effective PD programmes within a sector undergoing rapid 
change. The influence of other actions being implemented as part of the ECE Strategic Plan is also 
evident. The pressure to achieve qualified teacher targets, and a shortage of qualified relievers to 
call on to cover for staff on PD leave, has had the unintended effect of reducing qualified teachers’ 
access to professional development. 

In addition to the difficulties qualified teachers faced in accessing PD, two other issues relating 
to access are apparent: firsdy, rural and isolated services find it more difficult to access professional 
development and face greater time and travel costs. Secondly, access to PD is uneven across 
service types with kindergartens and home-based services having the greatest difficulties. The 
evaluation was not able to gather detailed information about the ability of care-givers in home- 
based services to access PD but the literature suggests that their involvement in PD is highly 
problematic (Lidington, 2000; White, 2003). Given the rapid growth of home-based services in 
New Zealand it is important that this issue is addressed. 

Whilst PD programmes are contributing to practitioners’ effectiveness in a number of areas 
the evaluation highlights the need for PD programmes that will assist services to develop bicultural 
pedagogical and organisational practices and understandings within a sociocultural paradigm so as 
to deliver Te Whariki as a bicultural curriculum. A dual approach of integration of bicultural 
perspectives within programmes that have a wider focus (for example, on assessment) together 
with programmes that specifically focus on bicultural understandings per se, is required. 

Similarly, a multi-pronged approach to strengthening programmes to support the effective 
teaching of children from diverse families is required. Expanding the use of, and making explicit, 
inclusive practices within PD programmes may help to increase the repertoires of teachers in this 
area. Such approaches are often able to be integrated within broader programme foci, particularly 
around curriculum, teacher interactions, and assessment. In addition, PD opportunities that 
address issues of diversity through a number of lenses (e.g., cultural, special education, family 
make-up) are required, given the evidence of increasing diversity within New Zealand early 
childhood services (Shuker and Cherrington, 2009). 


POSTSCRIPT 

Ministry of Education-funded programmes as described in this evaluation were curtailed at the end 
of 2009, along with the cessation of the Centres of Innovation programme. MOE-funded PD 
programmes are expected to resume from July 2010 with a more targeted emphasis on specific 
communities and foci. 
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NOTE 

Licensed, chartered early childhood services in New Zealand include kindergartens 
(predominately sessional, for children aged 3-4 years); playcentres (parent-led sessional services 
predominately for 2-4 year olds; home-based services (for children aged birth through 4 years); 
education and care (including full-day and sessional programmes for children aged from birth 
through 4 years, and including services with specific philosophies, e.g., Montessori, Maori 
Immersion, Pasifika [Pacific Island] language nests); and Te Kahanga Reo (whanau (family)-based 
services catering for children aged birth through five years which use and promote Maori language 
and cultural practices). 

The Education Review Office conducts regular reviews of all schools and licensed early 
childhood education services in New Zealand. 

m Current policy in New Zealand ties funding levels in teacher-led services to the percentage of 
qualified and registered teachers working with children during each hour that the programme is 
offered. If a qualified, registered teacher cannot be replaced by a qualified, registered reliever then 
the centre’s funding for that period will be reduced. 
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